Skip to content

fix(search): bound response bodies and detect DuckDuckGo HTTP 202 challenges - #44

Open
i386 wants to merge 2 commits into
qntx-labs:mainfrom
general-inference:fix-search-response-handling
Open

i386 wants to merge 2 commits into
qntx-labs:mainfrom
general-inference:fix-search-response-handling

Conversation

@i386

@i386 i386 commented Oct 27, 2026

Copy link
Copy Markdown

DuckDuckGo HTTP 202 responses currently reach the success path because the challenge check sits inside !status.is_success(). The existing empty-body test passes through a separate body heuristic, so it misses this bug. Classify 202 before checking success, and exercise it with an otherwise valid results page both with and without fallback enabled.

Search providers also collect entire response bodies before parsing. Add a shared bounded reader for HTML and JSON responses, with a configurable 2 MiB default through HttpConfig::with_max_response_bytes. It rejects oversized declared lengths and checks every received chunk before accumulation, including responses without a length header. Oversized bodies return a structured, non-retryable ResponseTooLarge error; callers can continue supplying their own HTTP client.

Regression coverage includes exact size boundaries, chunked transfer, JSON parse errors, error serialization, and provider-level enforcement without retries or DuckDuckGo fallback. The change also removes an unused DuckDuckGo dependency, gates feature-specific test imports, and updates one existing assertion for current stable Clippy.

Validation on macOS with the repository's stable toolchain:

  • cargo fmt --all --check
  • cargo check --workspace --all-features
  • cargo clippy --workspace --all-targets --all-features -- -D warnings
  • cargo build --workspace --all-features
  • cargo test --workspace --all-features — 305 passed, one existing ignored test
  • Default-feature search tests and strict Clippy
  • Strict library Clippy with each search provider enabled independently

The strengthened HTTP 202 regression was confirmed to fail before the fix. Tests use local HTTP fixtures; no live search-provider validation is claimed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant